Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic

This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite-state Markov process that modulates the arrival and service rates in the system. Various cases are considered: fast changing environment, fixed environment, and slowly changing environment. In all cases, the arrival rates are environment dependent, whereas the service rates are environment dependent when the environment Markov process is changing fast, and are assumed to be constant in the other two cases. In each of the cases, using weak convergence analysis, in particular functional limit theorems for Poisson processes and ergodic Markov processes, it is shown that an appropriate “averaged” version of the classical cμ-policy (the priority policy that favors classes with higher values of the product of holding cost c and service rate μ) is asymptotically optimal for an infinite horizon discounted cost criterion.